基于Spark SQL的分布式全文检索框架的设计与实现

随着信息化的深入,大数据在各个领域产生了巨大的价值,海量数据的存储和快速分析成为新的挑战。传统的关系型数据库由于性能、扩展性的不足以及价格昂贵等方面的缺点,难以满足大数据的存储和分析需求。Spark SQL是基于大数据处理框架Spark的数据分析工具,目前已支持TPC-DS基准,成为大数据背景下传统数据仓库的替代解决方案。全文检索作为一种文本搜索的有效方式,能够与一般的查询操作结合使用,提供更加丰富的查询和分析操作。目前,Spark SQL仅支持简单的查询操作,不支持全文检索。为了满足传统业务迁移和现有业务的使用需求,提出了分布式全文检索框架,涵盖了SQL文法、SQL翻译转换框架、全文检索并行化、检索优化4个模块,并在Spark SQL上进行了实现。实验结果表明相比于传统的数据库,在两种检索优化策略下,该框架的索引构建时间、查询时间分别减少到传统数据库的0.6%/0.5%和1%/10%,索引存储量减少为传统数据库的55.0%。     

A distributed command governor based on graph colorability theory

In this paper, a distributed command governor (CG) strategy is introduced that, by the use of graph colorability theory, improves the scalability property and the performance of recently introduced distributed noncooperative sequential CG strategies. The latter are characterized by the fact that only 1 agent at a decision time is allowed to update its command, whereas all the others keep applying their previously computed commands. The scalability of these early CG distributed schemes and their performance are limited because the structure of the constraints is not taken into account in their implementation. Here, by exploiting the idea that agents that are not directly coupled by the constraints can simultaneously update their control actions, the agents in the network are grouped into particular subsets (turns). At each time instant, on the basis of a round‐robin policy, all agents belonging to a turn are allowed to update simultaneously their commands, whereas agents in other turns keep applying their previous commands. Then, a turn‐based distributed CG strategy is proposed and its main properties are analyzed. Graph colorability theory is used to determine the minimal number of turns and to distribute each agent in at least a turn. A novel graph colorability problem that allows one to maximize the frequency at which agents can update their commands is proposed and discussed. A final example is presented to illustrate the effectiveness of the proposed strategy.     

Non‐planar projection performance evaluation and projector pose optimization

In projector applications, the actual display effect is primarily dependent on two aspects: projector performance parameters (intrinsic properties), and projector placement (extrinsic properties). Many studies have been conducted on the impact of the original resolution, illumination, and contrast ratio on projection performance. However, few studies have focused on the influence of extrinsic properties, including the position and orientation of the projector on the projection effect. In this paper, three projection performance evaluation criteria—projection resolution, resolution difference, and projection distance difference—are proposed in terms of extrinsic properties. Based on these evaluation criteria, a projection performance evaluation function was constructed. Through this function, as well as optimization employing the multi‐start and Monte Carlo methods, projector pose parameters corresponding to optimal projection effects can be obtained. Evaluations demonstrated that the proposed projection performance evaluation criteria could correctly describe the impacts of projector placement on projection effects. Projection systems arranged according to the projector pose optimization methods achieved good performance on screens in difference shapes with different numbers of projectors, where the resolution and display improved with no change of projector hardware parameters. The optimal projector poses calculated using these optimization methods can provide theoretical guidance for practical projector placement.     

Lift-and-project MVEE based convex hull for robust SCED with wind power integration using historical data-driven modeling approach

This paper presents an adjustable robust security constrained economic dispatch (SCED) model with wind power uncertainties. First, the scenario based adjustable robust SCED model is presented. It considers multiple scenarios from historical data as well as the spatial correlation among wind farms. Then, the proposed SCED model becomes an optimization problem with a large amount of constraints which is skillfully solved using a lift-and-project minimum volume enclosing ellipsoid (MVEE) based convex hull. Furthermore, the proposed model is transformed into a second order cone programming (SOCP) model by the use of participation factors to generate adjustable generation outputs and thus guarantee the energy balance. In order to further reduce the computational complexity, the inactive constraints reduction strategy is proposed to quickly eliminate inactive SOC security constraints before solving the model. Numerical results of IEEE 14-bus and 118-bus test systems as well as the practical Polish power systems with several wind farms show that the proposed model can achieve better economies. Moreover, more than 82% of security constraints are identified as inactive in various cases of the simulation, and the proposed inactive constraints reduction strategy is promising for improving the computational performance.     

An intelligent θ-Modified Bat Algorithm to solve the non-convex economic dispatch problem considering practical constraints

This paper proposes a practical formulation for the non-convex economic dispatch problem to consider multi-fuel options, ramp rate limits, valve loading effect, prohibited operating zones and spinning reserve. A new optimization algorithm based on the θ-bat algorithm (θ-BA) is suggested to solve the problem. The θ-BA converts the Cartesian search space into the polar coordinates such that more search ability would be achieved. According to the complex, nonlinear, and constrained nature of the problem, a new self-adaptive modification method is proposed. The proposed modified θ-BA (θ-MBA) is constructed based on the roulette wheel mechanism to effectively increase the convergence of the algorithm. The high ability and satisfying performance of the proposed optimization method is examined on IEEE 15-unit, 40-unit and 100-unit test systems.     

Hierarchical particle swarm optimizer for minimizing the non-convex potential energy of molecular structure

The stable conformation of a molecule is greatly important to uncover the secret of its properties and functions. Generally, the conformation of a molecule will be the most stable when it is of the minimum potential energy. Accordingly, the determination of the conformation can be solved in the optimization framework. It is, however, not an easy task to achieve the only conformation with the lowest energy among all the potential ones because of the high complexity of the energy landscape and the exponential computation increasing with molecular size. In this paper, we develop a hierarchical and heterogeneous particle swarm optimizer (HHPSO) to deal with the problem in the minimization of the potential energy. The proposed method is evaluated over a scalable simplified molecular potential energy function with up to 200 degrees of freedom and a realistic energy function of pseudo-ethane molecule. The experimental results are compared with other six PSO variants and four genetic algorithms. The results show HHPSO is significantly better than the compared PSOs with p-value less than 0.01277 over molecular potential energy function.     

A negotiation support system for resolving an international trans-boundary natural resource conflict

Post-Soviet legal governance regime of Caspian Sea – the largest inland body of water on earth – remains a source of conflict among the five coastal states of Azerbaijan, Iran, Kazakhstan, Russia, and Turkmenistan. Although different division methods have been suggested for sharing the sea and its valuable resources, the actual gain of the countries is unclear as the proposed methods focus either on the oil and gas or the areal share of the parties. The Caspian Sea Negotiation Support System (Caspian Sea NSS) is developed in this study to delineate optimal boundaries for sharing the sea through simultaneous consideration of the countries' areal and resource shares under different sharing methods. This NSS is a complex optimization model, with a solver engine that provides reliable results with a reasonable computational effort using a heuristic method. The model is run under different division scenarios to evaluate the sensitivity of each party's gain and locations of nautical boundaries to the division rules and the economic values of the resources. Results show a high sensitivity of the optimal nautical boundaries to the division rules and an indirect relationship between the allocated area and resource shares. The findings highlight the necessity for considering utility shares in negotiations as opposed to adopting areal division rules which ignore the utilities and might result in unfair resource allocation. The main policy implication of the study is that clarification of the countries' resource and areal gain under any suggested legal regime for governing the Caspian Sea is essential to the success of the negotiations.     

Distribution system feeder re-phasing considering voltage-dependency of loads

Phase-balancing creates voltage changes in the network which calls for incorporating voltage-dependency of loads in the process of phase-balancing. Hence, inclusion of voltage-dependency in current-injection based three-phase load flow is investigated and the results are compared with constant-power load model in terms of phase-balancing. The problem being combinatorial, application of particle swarm optimization is investigated for phase-balancing problem of radial distribution network. The effects of phase unbalance and load representation are studied in terms of various parameters. It is observed that there are situations that lead to increase in losses despite improvement in phase-balancing.     

基于遗传算法的格构增强复材夹芯板优化设计方法研究

格构增强复材夹芯板在土木工程领域已得到广泛应用,然而工程优化设计尚未得到解决。基于现有的格构增强复材夹芯板受力性能,提出了基于遗传算法的优化设计方法,并在此基础上对其建设成本进行优化计算。最后与已投入使用的格构增强泡沫复材夹芯板相比较,结果表明,在满足承载力的前提下,相比于原构件,优化后的构件建设成本显著降低,经济效益得以提升。     

Integrated multi-layer representation and ant colony search for product selective disassembly planning

Selective disassembly plays an important role in product life cycle to meet requirements of the product repairing, reusing and recycling. An efficient disassembly plan is essential to minimize processing time in product maintenance for cost saving. This paper introduces a method for integration of the multi-layer product representation and the optimal search in product selective disassembly planning. The multi-layer representation is based on the product structure formed in product design. The method enables an efficient search for the disassembly sequence. Unlike the existing product representation methods, the multi-layer representation is a dynamic product data model integrated with an ant colony search process for a near optimal solution. Industrial applications have proven the method effectiveness.